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Purpose: To clarify the effects of missing values due to behavioral and psycholog- 
ical symptoms in dementia (BPSD) in Alzheimer's disease (AD) patients on the 
neuropsychological tests, this study describes the pattern of missing values due to 
BPSD, and its influence on tests. Materials and Methods: Drug-naive probable 
AD patients (n=127) with BPSD and without BPSD (n=32) were assessed with 
Seoul Neuropsychological Screening Battery including measures of memory, in- 
telligence, and executive functioning. Moreover, patients were rated on Korean 
Neuropsychiatry Inventory (K-NPI). Results: The more severe the K-NPI score, 
the less neuropsychological tests were assessable, leading to many missing values. 
Patients with BPSD were more severely demented than those without BPSD. K- 
NPI scores were significantly correlated with the number of missing values. The 
effect of BPSD was largest for tests measuring frontal functions. The replacement 
of the missing values due to BPSD by the lowest observed score also showed the 
largest effect on tests of frontal function. Conclusion: The global cognitive and 
behavior scales are related with missing values. Among K-NPI sub-domains, delu- 
sion, depressing, apathy, and aberrant motor behavior are significantly correlated 
for missing values. Data imputation of missing values due to BPSD provides a 
more differentiated picture of cognitive deficits in AD with BPSD. 
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Alzheimer's disease (AD) is a progressive neurodegenerative disorder character- 
ized by declining memory and cognitive function. In addition to these cognitive 
symptoms, most patients suffer from neuropsychiatric symptoms called 'behavior- 
al and psychological symptoms of dementia (BPSD)'. Recently, BPSD have been 
increasingly recognized as an important part of the symptom of AD. They occur in 
80-90% of patients with AD, 1 2 affecting the quality of life of both patient and care- 
giver, resulting in change of patient's lifestyle and management. 3 

AD patients with BPSD tend to lack of concentration while taking cognitive 
tests, and may show low performance on these tests. Furthermore, AD patients 
with BPSD frequently refuse to take these tests. Consequently, all or some part of 



YONSEI MED J HTTP:/A/VWW. EYMJ.ORG VOLUME 54 NUMBER 4 JULY 201 3 



825 



Yong Tae Kwak, et al. 



neuropsychological tests may be left behind as missing val- 
ues in specific neuropsychological tests. Due to these prob- 
lems, it is difficult to complete all neuropsychological test 
battery in AD patients with BPSD. 

Generally, if missing values occur in considerable por- 
tions of a study, increasing the sample population may be 
an appropriate statistical option. However, if increasing the 
sample population is difficult due to rare occurrence of such 
cases, missing values may reduce the precision of calculated 
statistics because there is less information than originally 
planned. Furthermore, if the missing mechanism is not at 
random, the result can be misleading. Another concern is 
that the assumptions behind many statistical procedures are 
based on complete cases, therefore missing values can com- 
plicate the theory required. 

We studied herein the number of missing values of neu- 
ropsychological tests occurring in AD patients with BPSD, 
sub-domains of BPSD related with missing values and their 
influence on the profile of neuropsychological test results. 
Finally, after the imputation of these missing values, we 
compared observed and adjusted data for AD with BPSD 
and without BPSD. 



MATERIALS AND METHODS 



Patients 

An initial 1212 patients with dementia were screened from 
March 2003 to January 2012 at the Hyoja Geriatric Hospital. 
From this initial screening, 159 patients with probable AD 
(these patients were newly diagnosed and never medicated 
before visiting the hospital) were recruited to be the subjects 
of this study. Among them, 127 patients were suffering from 
BPSD and others had no BPSD. All patients included in this 
study met National Institute of Neurological and Communi- 
cative Disorders and Stroke- Alzheimer's Disease and Relat- 
ed Disorders Association criteria for probable AD. 4 The pa- 
tients were drug-naive, except for episodic hypnotics that 
were taken for sleep disturbances. Patients who were taking 
psychotropic drugs, including antipsychotics, antidepres- 
sants, anticonvulsants, benzodiazepines, and cholinesterase 
inhibitors, and not educated were excluded from this study. 

The diagnostic evaluation included complete medical his- 
tory, physical and neurological evaluation, comprehensive 
neuropsychological test, routine laboratory test, and brain 
magnetic resonance imaging or computed tomography scans. 
The age at the onset of dementia was defined as the time of 



onset of memory disturbances that exceeded episodic forget- 
fulness. After complete description of the study was given to 
subjects, and caregivers, written informed consent was ob- 
tained from the patients or caregivers. This study was ap- 
proved by the Local Ethics Committee. 

Neuropsychological examinations 

All study subjects underwent the Seoul Neuropsychological 
Screening Battery, 5 a neuropsychological battery that in- 
cludes standardized and validated tests of various cognitive 
domains. The test assesses attention, language, praxis, cal- 
culation, visuoconstructive function, verbal and visual 
memory, and frontal/executive functions and provides nu- 
meric scores in most test items. Among them, digit span 
(forward and backward), the Korean version of the Boston 
Naming Test, calculation, ideomotor praxis, the Rey-Oster- 
rieth Complex Figure Test (RCFT) (copying, immediate 
and 20-min delayed recall and recognition), the Seoul Ver- 
bal Learning Test (SVLT) (three learning-immediate recall 
trials of a 12 item list, a 20-min delayed recall trial for the 12 
items), contrasting program test/go-no-go test, a test of se- 
mantic fluency and letter-phonemic fluency (the Controlled 
Oral Word Association Test, or CO WAT), Stroop test (Col- 
or Word Stroop Test or CWST; correct number of responses 
for word reading and naming the color of the font for 112 
items during a two-min period) were adopted for this study. 
All participants underwent neuropsychological tests using 
the same protocol. When tests could not be completed for 
any reasons, we repeated the missed test in another day by 
the same examiner. Nevertheless, if we were confronted 
with missing values, then we distinguished between miss- 
ing values due to behavioral disturbances and missing val- 
ues due to other reasons (e.g. communication problem, 
medical illness etc.). Only missing values due to behavioral 
problems were the subjects of this study. 

Behavioral rating scale 

The Korean version of the NPI (K-NPI), 6 which is a retro- 
spective iriformant-based rating scale for the behavioral and 
psychological symptoms in patients with dementia, was 
used for the evaluation of neuropsychiatric symptoms. This 
scale addresses 12 specific behavioral and psychological 
symptoms; delusions, hallucinations, agitation-aggression, 
depression, anxiety, euphoria, apathy, disinhibition, irritabili- 
ty, aberrant motor behavior, sleep disturbances, and eating 
abnormalities. The K-NPI gives a composite score for each 
domain that is the product of frequency by severity of sub- 
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Table 1. Demographic Data for Drug-Naive AD Patients with BPSD (Mean±Standard Deviation) 





BPSD (-) 


BPSD (+) 


p value* 


Numbers (female) 


32 (17) 


127 (72) 


0.101 


Age (vrs) 


72.2±7.1 


74.6±7.6 


0.116 


Onset age (yrs) 


70.2±6.9 


71.7±8.5 


0.383 


Duration (months) 


29.2±18.5 


36.3±29.0 


0.193 


Education 


11.7±4.6 


9.5±5.1 


0.029 


K-MMSE 


21.3±3.5 


20.4±4.4 


0.197 


CDR 


0.73±0.4 


1.0±0.5 


0.008 


CDR-SB 


3.4±1.8 


5.1±2.8 


0.000 


Barthel 


19.9±0.4 


19.4±1.5 


0.001 


GDS 


12.8±7.6 


14.4±7.0 


0.280 



BPSD, behavioral and psychological symptoms in dementia; K-MMSE, Korean Mini-Mental State Examination; CDR, Clinical Dementia Rating Scale; CDR- 
SB, Clinical Dementia Rating Scale-Sum of Boxes; GDS, Geriatric Depression Scale; AD, Alzheimer's disease. 
'Independent t-test, chi-square test was done. 



scores. The total K-NPI score was calculated by adding the 
12 composite scores. The test-retest reliability and internal 
consistency of the K-NPI was assessed in a previous study. 6 
In order to assess global dementia severity, the Korean 
version of the Mini-Mental State Examination (K-MMSE), 7 
Clinical Dementia Rating Scale (CDR), 8 and the Clinical 
Dementia Rating Scale-Sum of Boxes (CDR-SB) were 
used. A Barthel index 9 for the activities of daily living evalu- 
ation, and a Geriatric Depression Scale (GDS) 10 for depres- 
sion were also assessed. 

Statistical analysis 

First, the baseline characteristics of the AD patients with and 
without BPSD were assessed by using Student's two-tailed t- 
tests and chi-square tests. Second, missing values in each 
neuropsychological test were assessed. Third, we computed 
Pearson's correlation coefficients between the K-NPI sub-do- 
mains and the number of missing values in individual neuro- 
psychological tests for significance using the two tailed tests. 

For all test scores, means were calculated. Next, while 
we replaced each missing value in AD without BPSD by 
series mean, in AD with BPSD, we replaced each missing 
value by the worst score found on that particular test. After 
this adjustment, we calculated means of the adjusted scores. 

A significance level of 0.05 (two-tailed) was set for all 
analyses, which were performed by using the SPSS version 
18.0 (SPSS Inc., Chicago, IL, USA) for Windows. 



RESULTS 



Table 2. List of Total Missing Value Related Demographic 
Variables with Corresponding Weights(B) and Probability 
Levels in Patients of Alzheimer's Disease 





B 


p value* 


Age 


-0.253 


0.649 


Onset age 


0.301 


0.590 


Duration 


0.042 


0.410 


Sex 


0.085 


0.890 


Education 


-0.029 


0.667 


K-MMSE 


-0.312 


0.002 


CDR 


-0.760 


0.461 


CDR-SB 


0.110 


0.560 


Barthel 


-0.111 


0.450 


GDS 


0.032 


0.429 



CDR, Clinical Dementia Rating Scale; CDR-SB, Clinical Dementia Rating 

Scale-Sum of Boxes; GDS, Geriatric Depression Scale. 

The number of total missing values significantly related with K-MMSE 

(Korean Mini-Mental State Examination). 

R=0.528, 'Multiple regression analysis was done. 

BPSD. The AD with BPSD showed significantly higher 
scores for CDR, and CDR-SB, and lower Barthel index than 
AD without BPSD. The education level of AD with BPSD 
was significantly lower than that of AD without BPSD (Ta- 
ble 1). Age, onset age, and duration were not significantly 
different between AD with BPSD and without BPSD. 

Demographic factors influencing total missing values 

Using multiple regression analysis, we searched for the de- 
mographic factors that influenced the number of total miss- 
ing values in study groups. The K-MMSE score was only 
statistically related to the number of total missing values 
(Table 2). 



Characteristics of the patients 

The study included 127 AD with BPSD and 32 AD without 



Number of missing values in study subjects 

Some patients were unable to perform certain neuropsycho- 
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logical tests and the number of missing values due to BPSD 
is shown in Table 3. Except CO WAT animal and supermar- 
ket, most missing values occurred more often in AD patients 
with BPSD than in AD patients without BPSD. Among neu- 
ropsychological tests, RCFT copy, RCFT immediate recall, 
RCFT delayed recall, CWST word and CWST color were 
more significantly unable to be completed by AD patients 
with BPSD (Table 3). 



test between AD patients with and without BPSD in ob- 
served and adjusted data. In observed data, AD patients with 
BPSD showed significantly lower scores in SVLT immedi- 
ate recall and the CWST word test. After data adjustment, 
AD patients with BPSD showed additionally significantly 
lower scores in COWAT phonemic, contrasting program 
and the CWST color test. In the CWST word test, statistical 
significance was increased more. 



Correlation of K-NPI sub-domains with missing values 
numbers of neuropsychological tests, CDR, disease 
duration and age in AD with BPSD 

Our patients were often unable to perform certain neuro- 
psychological tests. To clarify the relationship between K- 
NPI sub-domains and missing value numbers of specific 
neuropsychological tests, correlation among these parame- 
ters was tested, and the results are shown in Table 3. Delu- 
sions, depression, apathy, aberrant motor behavior, and total 
K-NPI scores were significantly correlated with the number 
of total missing values (Table 4). 

Differences of neuropsychological tests between AD 
with BPSD and without BPSD in observed and 
adjusted data 

Table 5 shows the differences of each neuropsychological 



DISCUSSION 



This study aimed to assess the prevalence of missing val- 
ues, relation between BPSD and missing values of the neu- 
ropsychological test, and value of missing values analysis 
in AD patients with BPSD. 

Most neuropsychological tests assessing various cognitive 
domains require some degree of attention and cooperation 
of patients. If examinees have behavior problems or have 
severe specific cognitive impairment, there is a high proba- 
bility of specific neuropsychological tests being left unfin- 
ished. For example, agitation, inability to correct, loss of in- 
sight into performance, perseverative behavior, stereotypy 
behavior, and stimulus boundedness were strongly related 
with missing data analysis in patients with frontotemporal 



Table 3. Number of Valid Test, Missing Value due to Behavior Disturbances (in AD With BPSD) and Other Reasons (in AD 
Without BPSD) 



Tests 




BPSD (-) 




BPSD (+) 


p value* 




Valid 


Missing value (%) 


Valid 


Missing value (%) 


Digit forward 


32 


0(0) 


124 


3 (2.4) 


0.507 


Digit backward 


32 


0(0) 


122 


5 (3.9) 


0.320 


K-BNT 


31 


1(3.1) 


122 


5 (3.9) 


0.652 


Calculation 


32 


0(0) 


121 


6 (4.7) 


0.253 


Ideomotor praxis 


32 


0(0) 


123 


4(3.1) 


0.403 


SVLT immediate recall 


32 


0(0) 


126 


1 (0.8) 


0.799 


SVLT delayed recall 


32 


0(0) 


126 


1 (0.8) 


0.799 


RCFT copy 


32 


0(0) 


107 


20 (15.7) 


0.014 


RCFT immediate copy 


32 


1(3.1) 


105 


22 (17.3) 


0.049 


RCFT delayed copy 


31 


1(3.1) 


105 


22(17.3) 


0.029 


Contrasting 


30 


2(6.3) 


117 


10 (7.9) 


0.551 


Go-no-go 


30 


2(6.3) 


116 


11 (8.7) 


0.492 


COWAT animal 


31 


1 (3.3) 


126 


1 (0.8) 


0.363 


COWAT supermarket 


31 


1 (3.3) 


126 


1 (0.8) 


0.363 


COWAT phonemic 


21 


11 (34.4) 


77 


50 (39.4) 


0.687 


CWST word correct 


28 


4(12.5) 


90 


37(29.1) 


0.040 


CWST color correct 


27 


5 (15.6) 


83 


44 (34.6) 


0.027 



BPSD, behavioral and psychological symptoms in dementia; K-BNT, Korean version of the Boston Naming Test; RCFT, Rey-Osterrieth Complex Figure Test; 
SVLT, Seoul Verbal Learning Test; COWAT, Controlled Oral Word Association Test; CWST, Color Word Stroop Test; AD, Alzheimer's disease. 
"Independent t-test test was done. 
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dementia. 11 

In contrast to this frontotemporal dementia study, our 
study showed that delusions, depression, apathy, and aber- 
rant motor behaviors were closely related with missing val- 
ues of test. Among them, delusions and aberrant motor be- 
haviors were more significantly correlated with missing 
values than other K-NPI sub-domains (Table 4). The reason 



for the close relationship between delusions with missing 
data is not clear. One possible explanation is that delusion is 
closely related with agitation and aberrant motor behav- 
ior, 1213 and these physical symptoms apparently lead to 
problems in assessing these time-consuming tests. More- 
over, patients with delusions frequently have persecutory 
delusion, 1415 and as a results some patients with this delu- 



Table 4. Correlation of K-NPI Sub-Domains with Number of Missing Values, Clinical Dementia Rating Scales, Disease Dura- 
tion and Age 





Number of total 
missing values 


K-MMSE 


CDR 




Duration 


Age 


Delusions 


0.359* 


-0.379* 


0.374* 




0.087 


0.128 


Hallucinations 


0.100 


-0.097 


0.181* 




0.042 


-0.040 


Aggression 


0.156 


-0.373* 


0.436* 




0.086 


0.032 


Depression 


0.201* 


-0.212* 


0.181* 




0.128 


0.083 


Anxiety 


0.133 


-0.144 


0.148 




0.023 


-0.036 


Euphoria 


0.058 


-0.008 


0.015 




-0.034 


-0.027 


Apathy 


0.148* 


-0.271* 


0.251* 




0.226* 


-0.046 


Disinhibition 


-0.027 


0.014* 


0.120 




0.152 


0.039 


Irritability 


0.043 


-0.105* 


0.254* 




0.126 


-0.081 


Motor behavior 


0.259* 


-0.402* 


0.462* 




-0.093 


-0.097 


Nighttime behavior 


0.125 


0.553* 


0.127 




0.124 


0.127 


Eating change 


-0.005 


0.487* 


0.082 




-0.084 


-0.030 


Total K-NPI score 


0.195* 


-0.518 


0.360* 




0.108 


-0.023 


K-NPI, Korean Neuropsychiatric Inventory; K-MMSE, Korean Mini-Mental State Examination; CDR, Clinical Dementia Rating Scale. 




*/>value <0.05. 














Value <0.01. 














Table 5. Observed and Adjusted Means Difference between AD Patients With BPSD and Without BPSD 








Observed data 






Adjusted data 






BPSD (-) 


BPSD (+) 


Difference 


BPSD (-) 


BPSD (+) 




Digit forward 


5.0 


4.8 


0.2 


5.0 


4.6 




Digit backward 


3.1 


2.9 


0.2 


3.1 


2.7 


0 4 


K-BNT 


31.8 


28.9 


2.9 


31.8 


27.7 


4.1 


Calculation 


8.6 


8.5 


0.1 


8.6 


8.1 


0.5 


Ideomotor praxis 


3.4 


3.5 


-0.1 


3.4 


3.4 


0.0 


SVLT immediate recall 


13.1 


11.2 


1.9* 


13.1 


11.1 


2.0* 


SVLT delayed recall 


1.6 


1.1 


0.5 


1.6 


1.1 


0.5 


RCFT copy 


21.6 


21.8 


-0.2 


21.6 


18.4 


3.2 


RCFT immediate copy 


3.2 


4.0 


-0.8 


3.1 


3.3 


-0.2 


RCFT delayed copy 


2.9 


3.3 


-0.4 


2.9 


2.7 


0.2 


Contrasting 


18.2 


15.9 


2.3 


18.2 


14.8 


3.4* 


Go-no-go 


13.1 


12.6 


0.5 


13.1 


11.6 


1.5 


COWAT animal 


9.8 


9.1 


0.7 


9.8 


9.0 


0.1 


COWAT supermarket 


9.8 


8.9 


0.9 


9.8 


8.8 


1.0 


COWAT phonemic 


14.5 


14.6 


-0.1 


14.5 


8.9 


5.6* 


CWST word correct 


106.0 


101.9 


-4.1* 


106.0 


72.8 


33.2* 


CWST color correct 


47.1 


43.9 


-3.2 


47.1 


29.2 


17.9* 



BPSD, behavioral and psychological symptoms in dementia; K-BNT, Korean version of the Boston Naming Test; RCFT, Rey-Osterrieth Complex Figure Test; 
SVLT, Seoul Verbal Learning Test; COWAT, Controlled Oral Word Association Test; CWST, Color Word Stroop Test; AD, Alzheimer's disease. 
*/>value <0.05. 
Value <0.01. 
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sion may resist diagnostic evaluations. 

The BPSD showed the largest effect on tests measuring 
the frontal functions (contrasting programs, CWST word, 
CWST color and COWAT phonemic). Missing values in 
these tests (sensitive markers for impairment of frontal func- 
tion) may blur the differences of cognitive sub-domain func- 
tions in AD with BPSD. In contrast, however, tests such as 
the SVLT immediate and delay recall (sensitive markers for 
temporal function) showed nearly no differences in missing 
values between AD with BPSD and without BPSD. 

The missing values were differently replaced in each group. 
According to missing value analysis, missing values in AD 
without BPSD were replaced by the mean scores. Howev- 
er, because the missing values of AD with BPSD were cor- 
related with K-NPI scores, these were replaced by the low- 
est observed values in AD with BPSD. The effect of the data 
imputation was largest for tests measuring frontal function. 
As a consequence, the difference between test results of 
frontal function becomes clear, and more differentiated pat- 
tern of cognitive deficits is found in AD with BPSD. 

In our study, the total missing data number was signifi- 
cantly related with K-MMSE scores, while global cognitive 
scale (CDR) was not (Table 2). These discrepancies may 
result from study population characteristics. Most of our 
study subjects were in the mild stage dementia. Therefore, 
CDR staging, which is only five grade scale, may not re- 
flect the differences. 

The relationship of neurocognitive functions in AD pa- 
tients with BPSD versus those without BPSD has been the 
subject of considerable debate. While some studies have 
noted difficulties in differentiating AD with and without 
BPSD on neuropsychological variables and have questioned 
the specificity of deficient sub-cognitive domains in AD pa- 
tients with BPSD compared with AD patients without 
BPSD, 16 other studies have shown a relationship with more 
cognitive impairment and frontal lobe dysfunctions. 1718 

Considering the fact that considerable portion of certain 
neuropsychological tests were left unfinished in our study, 
there is a possibility that missing values may have influ- 
enced our overall results, even though our study subjects 
were mainly mildly demented patients. Perhaps, the miss- 
ing values analysis might provide a clue for this finding. In 
this study, most patients without BPSD showed no difficul- 
ties in completing the neuropsychological test, therefore 
most neuropsychological tests were assessable, thus provid- 
ing the investigator with more complete data set. 

In a previous study, 19 the missing values (or non-response) 



on the mini mental state examination were explored. This 
study concluded that non-response did not occur randomly, 
and scoring non-response as error was more likely to reflect 
the severity of dementia. In randomized controlled clinical 
trial study, they showed that drop out (i.e. missing values) 
does not occur randomly, therefore the data can influence 
the overall results. 20 Due to these missing values, the neuro- 
psychological tests are biased toward the null effect. This 
might have obscured the characteristic cognitive deficit of 
the AD patients with BPSD. 

Nevertheless, replacing all missing values irrespective of 
its cause is not valid and feasible for all patient groups and 
has to be done with caution. Missing values may be caused 
by other reasons, such as sensory impairments, aphasia, or 
external factors caused by the examiner. In these cases, the 
failure to complete a specific neuropsychological test does 
not necessarily indicate a specific cognitive deficit, but is 
more likely to reflect other non-cognitive factors. 

In the present study, BPSD was found to be significantly 
correlated with missing values, therefore, we replaced only 
the missing values in AD patients with BPSD by the lowest 
observed scores. Missing values due to other reasons were 
excluded from this study. In our present study, therefore, re- 
placing missing values due to BPSD by the lowest scores is 
justified. 

However, this study had several limitations. First, sample 
size was relatively small compared to the number of factors. 
Second, this study included mostly milder dementia pa- 
tients. Since the BPSD was more prevalent in moderate to 
severe stage of dementia than mild stage of dementia, 21 
these methods were expected to be more applicable to mod- 
erate to severe stage of dementia groups. Third, even though 
we excluded the missing values due to other problems 
caused by examiner and other researcher, the possibility of 
false positive and negative cases could not completely be 
ruled out. Fourth, though K-MMSE score, which is only 
statistically influencing factor for the total missing data 
number, was not different between AD patients with and 
without BPSD, the patients with BPSD showed more signif- 
icant global cognitive dysfunctions. Therefore, there was 
possibility that these between-group differences might have 
originated from these severity differences. Finally, our 
study was a hospital-based study, therefore it may not rep- 
resent a real community. 

In conclusion, our results showed that missing values in 
the context of BPSD did not occur at random and provided 
useful information. Using data imputation, a more differen- 
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tiated picture of cognitive deficits in AD with BPSD is sug- 
gested without the need to exclude those patients who dis- 
played behavioral disturbances. Consequently, more realistic 
and reliable statistical analysis should be carried out and 
further research on the clinical picture of AD with BPSD 
and its relation to other BPSD would provide the answer. 
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